Update CONTRIBOP.MD FP16 support for MatMulIntegerToFloat #18751
Conversation
| |MultiHeadAttention|*in* query:**T**<br> *in* key:**T**<br> *in* value:**T**<br> *in* bias:**T**<br> *in* key_padding_mask:**M**<br> *in* relative_position_bias:**T**<br> *in* past_key:**T**<br> *in* past_value:**T**<br> *out* output:**T**<br> *out* present_key:**T**<br> *out* present_value:**T**|1+|**M** = tensor(int32)<br/> **T** = tensor(float), tensor(float16)| | ||
| |NhwcConv|*in* X:**T**<br> *in* W:**T**<br> *in* B:**T**<br> *out* Y:**T**|1+|**T** = tensor(float), tensor(float16)| | ||
| |QAttention|*in* input:**T1**<br> *in* weight:**T2**<br> *in* bias:**T3**<br> *in* input_scale:**T3**<br> *in* weight_scale:**T3**<br> *in* mask_index:**T4**<br> *in* input_zero_point:**T1**<br> *in* weight_zero_point:**T2**<br> *in* past:**T3**<br> *out* output:**T3**<br> *out* present:**T3**|1+|**T1** = tensor(int8), tensor(uint8)<br/> **T2** = tensor(int8), tensor(uint8)<br/> **T3** = tensor(float), tensor(float16)<br/> **T4** = tensor(int32)| |
There was a problem hiding this comment.
Hmm, are these other changes intended? It's supposed to be an update for MatMulIntegerToFloat, but why do I also see QAttention and QLinearConcat?
There was a problem hiding this comment.
Both files need to be updated, kernelDocumentation pipeline stage also verifies these file contents
There was a problem hiding this comment.
Both files need to be updated
I'm not disputing that we need them, but I am seeking to know why we're seeing these other changes. I'll presume we're seeing them because they are stale after a merge from main and sign off.
(note, if we ever generate this locally and see modifications under other providers, like CUDA, then we should not merge those since we don't build locally with CUDA)
| |MultiHeadAttention|*in* query:**T**<br> *in* key:**T**<br> *in* value:**T**<br> *in* bias:**T**<br> *in* key_padding_mask:**M**<br> *in* relative_position_bias:**T**<br> *in* past_key:**T**<br> *in* past_value:**T**<br> *out* output:**T**<br> *out* present_key:**T**<br> *out* present_value:**T**|1+|**M** = tensor(int32)<br/> **T** = tensor(float), tensor(float16)| | ||
| |NhwcConv|*in* X:**T**<br> *in* W:**T**<br> *in* B:**T**<br> *out* Y:**T**|1+|**T** = tensor(float), tensor(float16)| | ||
| |QAttention|*in* input:**T1**<br> *in* weight:**T2**<br> *in* bias:**T3**<br> *in* input_scale:**T3**<br> *in* weight_scale:**T3**<br> *in* mask_index:**T4**<br> *in* input_zero_point:**T1**<br> *in* weight_zero_point:**T2**<br> *in* past:**T3**<br> *out* output:**T3**<br> *out* present:**T3**|1+|**T1** = tensor(int8), tensor(uint8)<br/> **T2** = tensor(int8), tensor(uint8)<br/> **T3** = tensor(float), tensor(float16)<br/> **T4** = tensor(int32)| |
There was a problem hiding this comment.
Both files need to be updated
I'm not disputing that we need them, but I am seeking to know why we're seeing these other changes. I'll presume we're seeing them because they are stale after a merge from main and sign off.
(note, if we ever generate this locally and see modifications under other providers, like CUDA, then we should not merge those since we don't build locally with CUDA)
Description
Motivation and Context